detect/absent: Add error_or, must_error, and must_succeed options#15739
Closed
jlucovsky wants to merge 8 commits into
Closed
detect/absent: Add error_or, must_error, and must_succeed options#15739jlucovsky wants to merge 8 commits into
jlucovsky wants to merge 8 commits into
Conversation
SCInspectionBufferSetupAndApplyTransforms resets buffer->flags to 0, which clobbered the DCE byte-order flag when it was set first. Reorder so the byte-order flag is applied after the buffer is set up. Without this, dcepayload-15 (and similar tests that depend on the DCE byte-order flag reaching content inspection) fail. Issue: 7114
Issue: 7114 Add a way to flag inspection buffer errors. Add a new flag to track those.
Add parsing and engine analyzer support for the error_or, must_error, and must_succeed options to the absent keyword. Use enum DetectAbsentMode instead of individual bool fields. Validate that error_or/must_error/must_succeed require a failing transform, or_else rejects failing transforms, and must_error cannot be combined with other keywords. Issue: 7114
Issue: 7114 Mark from_base64 with SIGMATCH_TRANSFORM_CAN_FAIL and call SCInspectionBufferSetError when decode produces zero bytes.
Reset buffer->flags in InspectionBuffer setup functions so stale flags don't carry over across uses. Propagate buffer->flags into ciflags in detect-filename, detect-filemagic, detect-file-data, and detect-http-client-body so transform error flags reach content inspection. Issue: 7114
Document error_or, must_error, and must_succeed options for the absent keyword with examples and comparison of all five modes. Issue: 7114
Mark pcrexform with SIGMATCH_TRANSFORM_CAN_FAIL and call SCInspectionBufferSetError when the pcre match fails or capture group extraction fails. This enables error_or and must_error to detect pcrexform failures, and must_succeed to prevent false matches when the transform fails. Issue: 7114
Document that pcrexform signals an error when the pattern does not match, enabling use with absent: error_or, must_error, and must_succeed. Issue: 7114
jlucovsky
requested review from
jasonish,
jufajardini and
victorjulien
as code owners
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #15739 +/- ##
==========================================
- Coverage 82.96% 82.96% -0.01%
==========================================
Files 1003 1002 -1
Lines 275031 275307 +276
==========================================
+ Hits 228192 228418 +226
- Misses 46839 46889 +50
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
|
WARNING:
Pipeline = 32273 |
Contributor
Author
|
Continued in #15745 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Continuation of #15682
Add
error_orandmust_erroroptions to theabsentkeyword, enablingrules to detect when a transform operation (such as base64 decoding or
pcre matching) fails on an inspection buffer.
The five
absentmodes:absent— matches only when the buffer is absent (existing behavior)absent: or_else— matches when absent OR subsequent keywords match (existing behavior, renamed)absent: error_or— matches when a transform error occurs OR subsequent keywords matchabsent: must_error— matches only when a transform error occurs; no other keywords allowed on the same bufferPrevents false positives by ensuring content inspection only runs on successfully transformed data.
Link to ticket: https://redmine.openinfosecfoundation.org/issues/7114
Describe changes:
SCInspectionBufferSetError()to flag when a transform cannot complete with theDETECT_CI_FLAGS_ERRORflag. When this is set, the buffer data is not modified.from_base64andpcrexformsignal errors when the transform cannot complete.are paired with a can-fail transform; thator_elseis not paired with a can-fail transform (use error_or instead); thatmust_erroris not combined with other content keywords; thatabsentandfast_pattern` are not used on the same buffer; and that only one absent keyword appears per buffer.absent.modestringUpdates:
must_succeed-- matches on transform success only.Provide values to any of the below to override the defaults.
link to the pull request in the respective
_BRANCHvariable.SV_REPO=
SV_BRANCH=OISF/suricata-verify#2982
SU_REPO=
SU_BRANCH=